National Assessment of Educational Progress (NAEP)
Budget year: FY2003 Agency: 018
Part I: Summary Information and Justification o Section A: Overview o Section B: Summary of Spending o Section C: Acquisition/Contract Strategy o Section D: Performance Information o Section E: Security and Privacy o Section F: Enterprise Architecture (EA) Part II: Planning, Acquisition And Performance Information o Section A: Alternatives Analysis o Section B: Risk Management o Section C: Cost and Schedule Performance
Exhibit 300: Capital Asset Plan and Business Case Summary Part I: Summary Information And Justification Section A: Overview 1. Date of submission: Sep 10, 2007 2. Agency: 018 3. Bureau: 50
4. 5. 6. 7. 8.
Name of this Capital Asset: National Assessment of Educational Progress (NAEP) Unique Project (Investment) Identifier: 018-50-01-05-01-1020-00 What kind of investment will this be in FY2010? Mixed Life Cycle What was the first budget year this investment was submitted to OMB? FY2003 Provide a brief summary and justification for this investment, including a brief description of how this closes in part or in whole an identified agency performance gap: The National Center for Education Statistics (NCES) is the sponsoring entity for the National Assessment of Educational Progress (NAEP) program, a nationwide assessment effort involving multiple contractors responsible for performing the assessment and executing the vision of the assessment from NCES and the National Assessment Governing Board (NAGB). The execution of this vision includes: * The creation and coordination of test items and questionnaires to be used in a variety of assessment topics; * The development and production of test instruments and supporting materials for the assessments; * The hiring and management of full-time and parttime staff to coordinate and administer the assessments; * The creation and release of reports detailing the results of the assessments and the methodologies used to administer the assessments; and * The development and deployment of web-based applications to manage and integrate the contractor activities and to report results to the American Public. NCES and the Alliance contractors have established a set of tools to allow for the review and collaboration on project documentation, including project deliverables. These online tools open the communication among the parties involved in performing NAEP-related assessments and make critical documents accessible. The tools created for coordination include two critical systems for management. NAEP Network is a set of applications available to NCES, State NAEP coordinators, and NAEP assessment contractors to receive updated information and guidance regarding the current year's assessments and to collaborate with NAEP personnel. NAEP Network allows NCES, the Alliance contractors, and field staff to communicate and collaborate on status and activities of the current year's assessment. Additionally, the NAEP Integrated Management System (IMS) provides an online intranet for NCES and Alliance contractors, including a critical document repository, contact and calendar information, and other resources for the NAEP program. The IMS provides a central library of NAEP program documentation, including design plans, project deliverables, and schedules for review by NCES and Alliance contractors. The IMS also provides a secure environment for Alliance and NCES personnel to exchange information regarding the assessment efforts, to track document and plan changes, and to collaborate on document development.
9. Did the Agency's Executive/Investment Committee approve this request? yes a. If "yes," what was the date of this approval? Jun 14, 2007 10. Did the Project Manager review this Exhibit? yes 11. Contact information of Program/Project Manager?
a. What is the current FAC-P/PM (for civilian agencies) or DAWIA (for defense agencies) certification level of the program/project manager? Senior/Expert/DAWIA-Level 3 12. Has the agency developed and/or promoted cost effective, energy efficient and environmentally sustainable techniques or practices for this project. yes a. Will this investment include electronic assets (including computers)? yes b. Is this investment for new construction or major retrofit of a Federal building or facility? (answer applicable to non-IT assets only) no 1. If "yes," is an ESPC or UESC being used to help fund this investment? [Not answered] 2. If "yes," will this investment meet sustainable design principles? [Not answered] 3. If "yes," is it designed to be 30% more energy efficient than relevant code? [Not answered] 13. Does this investment directly support one of the PMA initiatives? yes Competitive Sourcing a. Briefly and specifically describe for each selected how this asset directly supports the identified initiative(s)? The NAEP Web Technology Operations program provides several management and oversight controls over the activities of the Alliance contractors in their execution of the NAEP program activities. These mechanisms keep NCES and Alliance partners informed of project progress and status as well as maintaining understanding of current issues or concerns. The three key areas of oversight including activity reporting, meeting coordination, and online collaboration. 14. Does this investment support a program assessed using the Program Assessment Rating Tool (PART)? (For more information about the PART, visit www.whitehouse.gov/omb/part.) no a. If "yes," does this investment address a weakness found during a PART review? [Not answered] b. If "yes," what is the name of the PARTed program? [Not answered] c. If "yes," what rating did the PART receive? [Not answered]
15. Is this investment for information technology? yes
For information technology investments only: 16. What is the level of the IT Project? (per CIO Council PM Guidance) Level 1 17. In addition to the answer in 11(a), what project management qualifications does the Project Manager have? (per CIO Council PM Guidance) (1) Project manager has been validated as qualified for this investment 18. Is this investment or any project(s) within this investment identified as "high risk" on the Q4-FY 2008 agency high risk report (per OMB Memorandum M-05-23)? no 19. Is this a financial management system? no a. If "yes," does this investment address a FFMIA compliance area? [Not answered] 1. If "yes," which compliance area: [Not answered] 2. If "no," what does it address? [Not answered] b. If "yes," please identify the system name(s) and system acronym(s) as reported in the most recent financial systems inventory update required by Circular A-11 section 52 [Not answered] 20. What is the percentage breakout for the total FY2010 funding request for the following? Hardware Software Services Other 2 3 95 0
21. If this project produces information dissemination products for the public, are these products published to the Internet in conformance with OMB Memorandum 05-04 and included in your agency inventory, schedules and priorities? yes 22. Contact information of individual responsible for privacy related questions:
23. Are the records produced by this investment appropriately scheduled with the National Archives and Records Administration's approval? yes 24. Does this investment directly support one of the GAO High Risk Areas? no
Section B: Summary of Spending 1.
Table 1: SUMMARY OF SPENDING FOR PROJECT PHASES (REPORTED IN MILLIONS)
(Estimates for BY+1 and beyond are for planning purposes only and do not represent budget decisions)
Planning: Acquisition: Subtotal Planning & 0 1.49 2.31 2.17 Acquisition: Operations & 10.82 3 2.62 2.43 Maintenance: TOTAL: 10.82 4.49 4.93 4.6 Government FTE Costs should not be included in the amounts provided above. Government FTE 0.33 0.11 0.12 0.12 Costs Number of FTE 0 1 1 1 represented by Costs: 2. Will this project require the agency to hire additional FTE's? no a. If "yes", How many and in what year? [Not answered] 3. If the summary of spending has changed from the FY2009 President's budget request, briefly explain
PY-1 and PY CY BY BY+1 earlier 2008 2009 2010 2011 0 0 0.58 0.35 0 1.49 1.73 1.82
BY+2 2012
BY+3 BY+4 and Total 2013 beyond
those changes: The spending has changes from the 2007 President's Budget Request to reflect recent direction from the National Assessment Governing Board (NAGB) regarding supporting requirements for IT management and for the expansion of the assessment scope and range. The adjustments to the budget reflect increased use of information technology (IT) in the conduct of assessments as well as increased functionality of management and reporting capabilities in current IT assets. These changes have resulted in increased requirements for IT solutions, including introduction of online questionnaires, electronic assessment instruments, and consolidated status reporting.
Section D: Performance Information
Performance Information Table
Fiscal Strategic Goal(s) Year Supported Goal 1: Objective 1: Improve Student Achievement in reading/language 2006 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in 2006 reading/language artsGoal 1: Objective 2: Improve Student
Measurement Measurement Area Grouping
Customer Results
Mission and Business Results
Measurement Baseline Target Indicator Average visitors to Integrated Management System per day to be Customer measured by number 50 80 Services of unique users (unique IP addresses) to the IMS as measured in user logs. Average number of page requests per month on NAEP web Frequency and site to be measured as 40,000 50,000 Depth an average of page requests to NAEP public web site and to
Actual Results
100
372,500
Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language Customer 2006 artsGoal 1: Results Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2006 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2006 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 2006 1: Improve Student
Customer Satisfaction
NAEP 2006 release web site. Percentage of NAEP data users who are satisfied or very satisfied with NAEP products. This 75 information will be provided as part of the NAEP Web Contractor's Award Fee Evaluation
100
100
Processes and Efficiency Activities
Integrated Management System (IMS) content items 15 increase with additional use.
25
54
Processes and Cycle Time Activities
Reduction in time required to review the 2 content to be published on the web.
3
4
Technology
Availability
Percentage of time NAEP web site is
92
95
97
Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2007 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2007 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in 2007 reading/language artsGoal 1: Objective 2: Improve Student
available. To be measured using uptime logs and access times recorded at host.
Customer Results
Customer Services
Average visitors to Integrated Management System per day to be measured by number 60 of unique users (unique IP addresses) to the IMS as measured in user logs.
85
65
Mission and Business Results
Average number of Frequency and page requests per 45,000 Depth month on NAEP web site
55,000 300,000
Customer Results
Customer Satisfaction
Percentage of NAEP data users who are satisfied or very satisfied with NAEP 75 products. This information will be provided as part of
100
100
Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language Processes and 2007 artsGoal 1: Efficiency Activities Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2007 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2007 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics 2007 Goal 1: Objective
Processes and Cycle Time Activities
the NAEP Web Contractor's Award Fee Evaluation. Integrated Management System (IMS) content items increase with additional use to be 10 measured in number of published pages and content items contained within the IMS. Reduction in time required to review content to be published on the web to be measured 1 against 2006 review cycles and 2007 review cycles using online review tools. Percentage of time NAEP web site is available to be measured using uptime logs and access times recorded at host.
20
20
2
2
Technology
Service Availability
94
97
96
Processes and Efficiency
Content Management 10
20
20
1: Improve Student Activities Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2008 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2008 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student 2008 Achievement in reading/language artsGoal 1:
Customer Results
Customer Services
System (CMS) and ADTracker content items increase with additional use, to be measured in number of published pages and content items contained within the CMS/ADTracker. Average visitors to Integrated Management System per day to be measured by number 75 of unique users (unique IP addresses) to the IMS as measured in user logs.
90
117
Mission and Business Results
Average number of page requests per month on NAEP web Frequency and site as measured by 200,000 350,000 339500 Depth total page requests from the NAEP public web site. Percentage of NAEP data users who are satisfied or very 80 satisfied with NAEP products. This
Customer Results
Customer Satisfaction
100
TBD
Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language Processes and 2008 artsGoal 1: Efficiency Activities Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2008 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2008 artsGoal 1: Objective 2: Improve Student Achievement in
Processes and Cycle Time Activities
Processes and Efficiency Activities
information will be provided as part of the NAEP Web Contractor's Award Fee Evaluation. Integrated Management System (IMS) content items increase with additional use to be 15 measured in number of published pages and content items contained within the IMS. Reduction in time (days) required to review content to be published on the web to be measured 2 against 2006 review cycles and 2007 review cycles using online review tools. Content Management System (CMS) and ADTracker content items increase with 15 additional use, to be measured in number of published pages and content items
25
34
4
4
25
25
Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in
contained within the CMS/ADTracker. Percentage of time NAEP web site is available to be measured using up95 time logs and access times recorded at host as hours available per month. Average visitors to Integrated Management System per day to be measured by number 80 of unique users (unique IP addresses) to the IMS as measured in user logs.
2008
Technology
Service Availability
99
99
2009
Customer Results
Customer Services
95
TBD
2009
Mission and Business Results
Average number of page requests per month on NAEP web Frequency and site as measured by 250,000 400,000 TBD Depth total page requests from the NAEP public web site. Customer Satisfaction Percentage of NAEP data users who are 90 satisfied or very
2009
Customer Results
100
TBD
reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2009 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2009 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in 2009 reading/language artsGoal 1: Objective 2: Improve Student
Processes and Efficiency Activities
Processes and Cycle Time Activities
Processes and Efficiency Activities
satisfied with NAEP products. This information will be provided as part of the NAEP Web Contractor's Award Fee Evaluation. Reduction in time (in days) required to review content to be published on the web to be measured 3 against 2008 review cycles and 2009 review cycles using online review tools. Reduction in time (in days) required to review content to be published on the web to be measured 3 against 2008 review cycles and 2009 review cycles using online review tools. Content Management System (CMS) and ADTracker content items increase with 18 additional use, to be measured in number of published pages
5
TBD
5
TBD
30
TBD
Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student
and content items contained within the CMS/ADTracker. Percentage of time NAEP web site is available to be measured using up96 time logs and access times recorded at host as hours available per month. Average visitors to Integrated Management System per day to be measured by number 80 of unique users (unique IP addresses) to the IMS as measured in user logs.
2009
Technology
Service Availability
99
TBD
2010
Customer Results
Customer Services
95
TBD
2010
Mission and Business Results
Average number of page requests per month on NAEP web Frequency and site as measured by 250,000 400,000 TBD Depth total page requests from the NAEP public web site. Customer Satisfaction Percentage of NAEP 90 data users who are 100 TBD
2010
Customer Results
Achievement in reading/language artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2010 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2010 artsGoal 1: Objective 2: Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in 2010 reading/language artsGoal 1: Objective 2:
Processes and Efficiency Activities
Processes and Cycle Time Activities
Processes and Efficiency Activities
satisfied or very satisfied with NAEP products. This information will be provided as part of the NAEP Web Contractor's Award Fee Evaluation. Reduction in time (in days) required to review content to be published on the web to be measured 3 against 2009 review cycles and 2010 review cycles using online review tools. Reduction in time (in days) required to review content to be published on the web to be measured 3 against 2009 review cycles and 2010 review cycles using online review tools. Content Management System (CMS) and ADTracker content 18 items increase with additional use, to be measured in number
5
TBD
5
TBD
30
TBD
Improve Student Achievement in Mathematics Goal 1: Objective 1: Improve Student Achievement in reading/language 2010 artsGoal 1: Technology Objective 2: Improve Student Achievement in Mathematics
of published pages and content items contained within the CMS/ADTracker. Percentage of time NAEP web site is available to be measured using up96 time logs and access times recorded at host as hours available per month.
Service Availability
99
TBD
Section F: Enterprise Architecture (EA) 1. Is this investment included in your agency's target enterprise architecture? yes a. If "no," please explain why? [Not answered] 2. Is this investment included in the agency's EA Transition Strategy? yes a. If "yes," provide the investment name as identified in the Transition Strategy provided in the agency's most recent annual EA Assessment. National Assessment of Educational Progress (NAEP) b. If "no," please explain why? [Not answered] 3. Is this investment identified in a completed and approved segment architecture? yes a. If "yes," provide the six digit code corresponding to the agency segment architecture. The segment architecture codes are maintained by the agency Chief Architect. For detailed guidance regarding segment architecture codes, please refer to http://www.egov.gov/. 069-000 4. Service Component Reference Model (SRM) Table : Agency FEA SRM FEA SRM Service Component
Agency
Internal
BY
Component Name
Component Description
Service Type
Component
Reused Component UPI Name
or Funding External Percentage Reuse?
Market Branding - Continued work will be made towards the establish of a "NAEP" or "Nation's Report NAEP Web Card" branding, Branding to include common lookand-feel across all NAEP-related products and easier access to NAEP tools. NAEP Customer Relationship Management The deployment and maintenance of the Customer Relationship NAEP CRM Management (CRM) suite of applications will be in place and supported. Additional support for the
Content Content [Not Publishing and Management answered] Delivery
[Not No answered] Reuse
3
Customer Product Relationship Management Management
Quality [Not No Management answered] Reuse
6
Enhanced Project Tracking
public's inquiries and the ability to provide selfservice to many of the questions facing the public and facing NAEP State Coordinators will be provided with the addition of a communications tracking system, a knowledgebase, and a participation tracking system. Enhanced Project Tracking - The NAEP project teams have worked to use current intranet systems to exchange project Tracking and Process planning and Workflow Tracking tracking documentation. However, the need remains to provide a single source of project status
Program / [Not No Project answered] Reuse Management
2
information. The NAEP contractor will provide a mechanism for providing snapshots of project progress to NCES and other appropriate participants in the NAEP program. The system will provide visibility into the status of the assessments and the tasks awaiting completion. New interface and content presentation through a redesigned Public Web Site in NAEP compliance with Knowledge Knowledge Content new NCES Distribution Management Engineering template and Delivery standards. This redesign will result in streamlined access to content within the system
Customer / [Not No Account answered] Reuse Management
8
and compliance with both revised NCES style and template guidelines and with Federal goals for metadata format and publication. Increased analytics delivered regarding NAEP results through the implementation of an enhanced knowledge query NAEP Data database system. Analytics This system will give access to the NAEP data to a secure list of personnel and will provide insights into state performance trends in NAEP. Increased program Task/Budget oversight through Tracking the implementation
Knowledge Discovery
Data Mining
Governance / [Not No Policy answered] Reuse Management
4
ActivityFinancial Based Management Management
Program / [Not No Project answered] Reuse Management
2
of a Program Task/Budget Tracking System to aid in the tracking and forecasting of tasks and expenditures for each NAEP assessment. Integrated NAEP Item Bank providing singleIntegrated view of item NAEP Item development Bank tools provided by NAEP Item Development Contractors National Indian Education Study NIES Web (NIES) Public Release Release Site Development and Support Maintenance and operational Operations support of and deployed Maintenance applications and software for the NAEP program.
Knowledge Information Management Sharing
Quality [Not No Management answered] Reuse
13
Content Content Quality [Not No Publishing and Management Management answered] Reuse Delivery
12
Development Enterprise and Application Integration Integration
Software [Not No Development answered] Reuse
45
Includes software updates, additions, and hosting services. Additional technical and functional support provided. Security management and access control Security and Identification services to Security Internal Access and manage access to Management Controls Control Authentication the NAEP web assets and applications.
[Not No answered] Reuse
5
FEA SRM Component Enterprise Application Integration Identification and Authentication Information Sharing Product Management
5. Technical Reference Model (TRM) Table: FEA TRM FEA TRM FEA TRM Service Service Area Service Standard Category Service Access and Delivery Service Access and Delivery Service Access and Delivery Service Access and Delivery Access Channels Service Requirements Service Requirements Other Electronic Channels Legislative / Compliance
Service Specification
Network Solutions (URL)
Accessibility Standards (508); Privacy Act Dedicated hosting provided by Hosting sub SMTP using Microsoft Internet Supporting Service Transport Information Server; LDAP using Network Services Microsoft Windows Server;
Enterprise Application Integration Enterprise Application Integration
Service Access and Delivery
DNS using Network Solutions HTTP/HTTPS using Microsoft Internet Information Server; FTP Service Transport Service Transport using Microsoft Windows Server Support Platforms Dependent Platform Microsoft Windows Server; Microsoft Windows ASP.NET Microsoft Internet Information Server Microsoft Internet Information Server Microsoft SharePoint Microsoft Content Management Server Microsoft Visual Studio.NET provided by sub Microsoft Visual SourceSafe provided by sub Microsoft Visio Microsoft SQL Server
Service Platform and Infrastructure Service Platform Information and Sharing Infrastructure Service Platform Information and Sharing Infrastructure Knowledge Service Platform Distribution and and Delivery Infrastructure Content Service Platform Publishing and and Delivery Infrastructure Service Platform Activity-Based and Management Infrastructure Enterprise Service Platform Application and Integration Infrastructure Service Interface Process Tracking and Integration Enterprise Service Platform Application and Integration Infrastructure
Delivery Servers Web Servers Application Servers
Delivery Servers
Delivery Servers Portal Servers Application Servers Integrated Development Environment Software Configuration Management Modeling Database
Delivery Servers Software Engineering Software Engineering Software Engineering Database / Storage
Enterprise Application Integration Identification and Authentication Content Publishing and Delivery Product Management Content Publishing and Delivery Enterprise Application Integration Enterprise Application Integration Knowledge Distribution and Delivery Information Sharing Information Sharing
Service Platform Hardware / and Infrastructure Infrastructure Component Security Framework Component Framework Component Framework Component Framework Component Framework
Servers / Computers
Dell Poweredge Server provided by sub
Certificates / Digital Certificate by Thawte Digital Signatures provided by sub Microsoft Internet Information Server Microsoft ASP and Microsoft ASP.NET
User Presentation Static Display / Interface User Presentation Dynamic Server/ Interface Side Display
User Presentation Microsoft Internet Information Content Rendering / Interface Server Business Logic Dependent Platform Database Connectivity Reporting and Analysis Data Format / Classification Data Transformation Application Servers Content Rendering Microsoft ASP and Microsoft ASP.NET Microsoft SQL Server
Service Platform Data and Management Infrastructure Component Framework Data Management Interoperability Interoperability Data Management Data Management
Microsoft SQL Server Microsoft Visual Studio for XML creation Microsoft Visual Studio for XML DTDs Microsoft SharePoint 2003 Portal Server Microsoft Content Management Server 2000
Service Interface and Integration Service Interface and Integration Service Platform Process Tracking and Infrastructure Service Platform Process Tracking and
Infrastructure 6. Will the application leverage existing components and/or applications across the Government (i.e., USA.Gov, Pay.Gov, etc)? no a. If "yes," please describe. [Not answered] Part II: Planning, Acquisition And Performance Information Section A: Alternatives Analysis 1. Did you conduct an alternatives analysis for this investment? yes a. If "yes," provide the date the analysis was completed? Sep 13, 2007 b. If "no," what is the anticipated date this analysis will be completed? [Not answered] c. If no analysis is planned, please briefly explain why: [Not answered]
2. Alternatives Analysis Results: Alternative Analyzed Risk Adjusted Lifecycle Costs estimate Risk Adjusted Lifecycle Benefits estimate
Description of Alternative
Alternative 1 provides for the use of a performance-based contract for web hosting and development services with oversight from NCES. The performance based contract provides for the contractor to serve as the primary developer of Alternative 1 web applications and software solutions to support the NAEP 28.97 program and its activities, including random sampling, data collection, processing large amounts of information, and publishing printed materials. The NCES commitment in this alternative is to provide manageria
7.78
Contractors would continue to perform such duties as described in Alternative 4, with the exception of performing or maintaining web operations. New federal employees would be hired to implement all functions of web operations and Alternative 2 maintenance. The contractor will be responsible for all 34.987 software development and software testing, but all hosting services including hosting management configuration management, and testing services will be provided by the new federal employees under this alternative Keep all business functions in-house by hiring more federal workers. New federal employees would be hired to perform all Alternative 3 33.26 NAEP activities listed in Alternative 1, thereby eliminating all contractor support. Status quo whereby current contractor remains performing Alternative 4 maintenance and support efforts and no additional costs for 37.22 increased functionality and increased services
0
0
0
3. Which alternative was selected by the Agency's Executive/Investment Committee and why was it chosen? Alternative 1 was selected because it offers the Government the best opportunity for success at the best possible value. The combination of an externally hosted web environment for secure intranet activities and data processing increases the efficiency of the other NAEP support contractors while reducing the overall managerial workload of DoED personnel. The use of contractor support in a performance-based model has provided and will provide the program with the best value for support and services to the NAEP program. Alternatives 2 and 3 limit the capabilities and potential for the project and thus affect the legislative time constraints of the project. By having elements of the system securely hosted outside of the DoED infrastructure, the program is able to perform rapid development of intranet applications, perform technology upgrades faster, and respond to changing requirements for information technology support and services more quickly. Additionally, Alternatives 2 and 3 would require significant investments by DoED in infrastructure and manpower to support the current NAEP web operational environment. Alternative 4 was not chosen for the alternative does not provide for the advancement of new technologies nor for increased streamlining efforts. Efforts performed here will require maintenance and support of existing systems and will result in
outdated applications during the current lifecycle. The overall cost of the effort will increase under this scenario as the skills required to maintain and support the existing systems become less available and more expensive to the program a. What year will the investment breakeven? (Specifically, when the budgeted costs savings execced the cumulative costs.) 2021 4. What specific qualitative benefits will be realized? Using the performance-based model will create the incentives to meet the established legislative milestones for reporting of NAEP results as well as ensure high customer satisfaction with project deliverables. Through the alternative selected, additional benefits will be derived and are planned for the effort. These benefits include: · Increased collaboration through the implementation of new collaboration features and more dynamic intranet activities among the NAEP Alliance contractors, assessment field staff, and DoED personnel. · Streamlined decision reviews of assessment reports and web content through the implementation of new content systems. · Reductions in labor efforts and printing costs through the use of online assessment and questionnaire instruments. 5. Federal Quantitative Benefits ($millions): Budgeted Cost Cost Savings Avoidance PY-1 and Prior PY CY BY 1.17 0 Cost savings resulting from improvements in program processes, resulting in reduced labor [Not answered] requirements for performing NAEP-related activities. Increased use of new technologies resulting in reductions in labor efforts required to perform [Not answered] NAEP-related activities. Increased use of new technologies resulting in [Not answered] Justification for Budgeted Cost Savings Justification for Budgeted Cost Avoidance
BY+1 BY+2
1.19 1.09
0 0
reductions in labor efforts required to perform NAEP-related activities. BY+3 BY+4 and Beyond Total LCC 3.45 Benefit
0
LCC = Life-cycle cost
6. Will the selected alternative replace a legacy system in-part or in-whole? no a. If "yes," are the migration costs associated with the migration to the selected alternative included in this investment, the legacy investment, or in a separate migration investment? [Not answered] b. If "yes," please provide the following information: List of Legacy Investment or Systems Name of the Legacy Investment or Systems UPI if available Date of the System Retirement There are no Legacy Investment or Systems.
Section B: Risk Management (All Capital Assets) 1. Does the investment have a Risk Management Plan? yes a. If "yes," what is the date of the plan? Aug 29, 2008 b. Has the Risk Management Plan been significantly changed since last year's submission to OMB? yes c. If "yes," describe any significant changes: Updated risk management plan for 2009 program year to reflect identified risks. Risks identified for 2009 include: * Loss of partial or total budget funding (Low probability/High impact) with mitigation towards creating management reserve. * Changes in technology standards (mid probability/mid impact) with mitigation in monitoring new technologies and adopting open standards in development. * Changes in customer requirements (low probability/mid impact) with
mitigation in early sign off on requirements and early involvement of users in development. * Increased security threats (mid probability/ mid impact) with mitigation to improving intrusion detection and surveillance of systems. * Changes in policies result in changed develpment direction (low probability/mid impact) with mitigation towards monitoring policy discussions. * Government staff changes (mid probability/low impact) with mitigation of education and documentation for project assets and activities. 2. If there currently is no plan, will a plan be developed? [Not answered] a. If "yes," what is the planned completion date? [Not answered] b. If "no," what is the strategy for managing the risks? [Not answered] 3. Briefly describe how investment risks are reflected in the life cycle cost estimate and investment schedule: Investment risks are reflected in the life cycle through the development of phase deployments and specific targets for deliverables to minimize risks of cost overruns and delivery issues.
Section C: Cost and Schedule Performance (All Capital Assets) 1. Does the earned value management system meet the criteria in ANSI/EIA Standard - 748? yes 2. Is the CV% or SV% greater than ± 10%? (CV%= CV/EV x 100; SV%= SV/PV x 100) no a. If "yes," was it the? [Not answered] b. If "yes," explain the causes of the variance: [Not answered] c. If "yes," describe the corrective actions [Not answered] 3. Has the investment re-baselined during the past fiscal year? no a. If "yes," when was it approved by the agency head? [Not answered]
4. Comparison of Initial Baseline and Current Approved Baseline: Current Baseline Initial Baseline Current Baseline Variance Description of Milestone Planned Total Completion Total Cost Schedule:Cost Percent Completion Cost ($M) Date ($M) (# days:$M) Complete
Date 36 FY2006 Project Coordination Activities 37 FY2006 NAEP Network Redesign 38 FY2006 NAEP Public Website Activities 39 FY2006 Quality Control/Quality Assurrance Activities 40 FY2006 Web Development Activities 41 FY2006 Documentation and Training Activities 42 FY2006 Additional Maintenance 43 FY006 Training Support for NAEP Sep 30, 2006 Sep 30, 2006 Sep 30, 2006
Estimated Planned:Actual Planned:Actual 0.19 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 Sep Sep 30, 30, 2006 2006 0.19 0.19 0 [Not 100 answered] [Not 100 answered] [Not 100 answered]
0.15
0.15 0.15
0
0.96
0.96 0.96
0
Sep 30, 2006
0.47
0.47 0.47
0
[Not 100 answered]
Sep 30, 2006 Sep 30, 2006 Sep 30, 2006 Sep 30, 2006
0.47
0.47 0.47
0
[Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered]
0.47
0.47 0.47
0
0.38
0.38 0.38
0
0.6
0.6
0.6
0
44 FY07 NAEP Sep 30, Maintenance 2007 45 FY07 Project Sep 30, Coordination 2007 Activities 47 FY07 Quality Control/Quality Sep 30, Assurance 2007 Activities 50 FY07 Sep 30, Application 2007 Maintenance 51 FY07 Training Sep 30, Support for 2007 NAEP 52 FY07 NIES Maintenance Sep 30, 2007
1.39
0.2
Sep 30, 2007 Sep 30, 2007
Sep 30, 2007 Sep 30, 2007
1.39 1.385
0
[Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered]
0.2
0.197
0
0.24
Sep Sep 30, 30, 2007 2007 Sep Sep 30, 30, 2007 2007 Sep Sep 30, 30, 2007 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007
0.24 0.424
0
0.36
0.36 0.665
0
0.3
0.3
0.098
0
0.29
0.29 0.007
0
46 FY07 NAEP Sep 30, Security 2007 48 FY07 NAEP Sep 30, DME Activities 2007 49 FY07 NAEP Sep 30, Branding/Design 2007 53 FY07 NAEP Sep 30, CRM 2007
0.37
0.37 0.05
0
1.74
1.74 1.571
0
0.29 0.23
0.29 0.025 0.23 0.371
0 0
Development 54 FY07 NAEP Sep 30, Project Tracking 2007 55 FY07 NAEP Public Site Redesign 56 FY07 NAEP Knowedge Query System 57 FY07 NAEP Program/Budget Tracking Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 0.18
0.43
0.35
0.18
58 FY07 NAEP Sep 30, Item Bank 2007 60 FY08 NAEP Sep 30, Maintenance 2008 61 FY08 Task 7.1 - Support Ongoing Services 62 FY08 NAEP Task 7.3 Training and Demonstrations 63 NAEP FY08 Task 7.5 Project Management Sep 30, 2008
0.1
2.558
2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2008
Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007 Sep 30, 2007
0.18 0.26
0
[Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 100 answered] [Not 76 answered] [Not 75 answered]
0.43 0.781
0
0.35 0.032
0
0.18 0.101
0
0.1
0.001
0
[Not 2.558 1.106 answered]
0
2.052
Sep [Not 30, 2.052 0.754 answered] 2008 Sep [Not 30, 0.111 0.03 answered] 2008 Sep [Not 30, 0.341 0.232 answered] 2008
0
Sep 30, 2008
0.111
0
[Not 65 answered]
Sep 30, 2008
0.341
0
[Not 80 answered]
64 NAEP FY08 Task 7.6 Sep 30, Alliance 2008 Coordination 65 FY08 NAEP Sep 30, Security 2008 65a NAEP FY08 NAEP Task 7.1.2.3 Technical Assistance 65b NAEP FY08 Task 7.1.2.4 Monitor Web Security 65c NAEP FY08 Task 7.4 Web Hosting 66 NAEP FY08 Task 7.2 - DME Activities 67 NAEP FY08 Task 7.2 Enhance and Expand NAEP Programs
0.054
Sep [Not 30, 0.054 0.09 answered] 2008 Sep [Not 30, 0.14 0.0885 answered] 2008 Sep [Not 30, 0.05 0.046 answered] 2008
0
[Not 80 answered]
0.14
0
78
0
Sep 30, 2008
0.05
0
[Not 80 answered]
Sep 30, 2008
0.01
Sep [Not 30, 0.01 0 answered] 2008 Sep 30, 2008 Sep 30, 2008 [Not 0.08 0 answered] [Not 1.38 1.309 answered]
0
[Not 80 answered]
Sep 30, 2008 Sep 30, 2008
0.08
0
[Not 75 answered] [Not 80 answered]
1.38
0
Sep 30, 2008
1.38
Sep [Not 30, 1.38 1.309 answered] 2008 Sep [Not 30, 2.29 0 answered] 2009
0
[Not 80 answered]
71 FY09 NAEP Sep 30, Maintenance 2009
2.29
0
[Not 0 answered]
72 FY09 Project Coordination Activities 73 FY09 NAEP Quality Control/Quality Assurance Activities 74 FY09 NAEP Application Maintenance 75 FY09 NAEP Training Support
Sep 30, 2009
0.21
Sep [Not 30, 0.21 0 answered] 2009 Sep [Not 30, 0.26 0 answered] 2009 Sep 30, 2009 Sep 30, 2009 Sep 30, 2009 Sep 30, 2009 Sep 30, 2009 Sep 30, 2009 Sep 30, 2009 Sep 30, [Not 1.51 0 answered] [Not 0.31 0 answered] [Not 0.42 0 answered] [Not 1.73 0 answered] [Not 0.26 0 answered] [Not 0.86 0 answered] [Not 0.46 0 answered] [Not 0.2 answered] 0
0
[Not 0 answered]
Sep 30, 2009
0.26
0
[Not 0 answered]
Sep 30, 2009 Sep 30, 2009
1.51
0
[Not 0 answered] [Not 0 answered] [Not 0 answered] [Not 0 answered] [Not 0 answered] [Not 0 answered] [Not 0 answered] [Not 0 answered]
0.31
0
76 FY09 NAEP Sep 30, Security 2009 77 FY09 NAEP Sep 30, DME Activities 2009 78 FY09 NAEP Sep 30, Public Site 2009 Redesign 79 FY09 NAEP Sep 30, Science ICT 2009 80 FY09 NAEP Sep 30, IMS Upgrades 2009 81 FY09 NAEP Sep 30, Online 2009
0.42
0
1.73
0
0.26
0
0.86
0
0.46 0.2
0 0
Questionnaires FY10 NAEP Maintenance FY10 NAEP Security FY10 NAEP DME Sep 30, 2010 Sep 30, 2010 Sep 30, 2010 2.35
0.44
1.78
2009 Sep 30, 2010 Sep 30, 2010 Sep 30, 2010
[Not [Not [Not [Not 2.35 0 answered] answered] answered] answered] [Not [Not [Not [Not 0.44 0 answered] answered] answered] answered] [Not [Not [Not [Not 1.78 0 answered] answered] answered] answered]